Run checks in Spacelift
This page runs the same ctfkit by compliance.tf checks on every Spacelift stack, reported as warnings first and enforced one stack at a time with a label.
ctfkit runs inside your stacks as two hooks and one plan policy. The hooks write small JSON files that Spacelift passes to the policy, and the policy turns them into a warning or a denial on the run.
Status: preview. The hooks and the policy are tested offline against the policy fixtures and the hook scripts, but the integration has not yet been checked on a live Spacelift run. In particular, the name of the saved plan file that the after_plan hook looks for is not confirmed. Start in shadow mode and read the first runs' logs before you enforce anything.
Get access
ctfkit is available to compliance.tf customers on request: see Get access. The files on this page are adapters/spacelift/config.stack.yml, config.yml and policy.rego in compliancetf/ctfkit, inlined here so the page stands alone.
- You build a runner image with ctfkit, your settings and your exception registry in it.
- Set both hooks in stack settings, never in the stack repository.
- Enforce one stack at a time with the stack label
ctfkit:enforce. Remove the label to roll back.
What the integration does
ctfkit reportreads thecompliancetf-manifest.jsonfile that each compliance.tf served module carries in.terraform/modules, and reports module instances that compliance.tf does not serve, builds whose manifest records a degraded posture, and controls delegated to a scope that no module in the configuration governs.ctfkit scanreads the plan JSON and reports raw resources that a compliance.tf module covers, providers outside your allowlist, missing tags and sensitive changes.- Each file carries a
verdict:failwhen any unsuppressed error-level finding exists, elsepass. The policy reads the verdict and never counts findings itself. It reads the two files frominput.third_party_metadata.custom.ctfkitandinput.third_party_metadata.custom["ctfkit-scan"]. - The hooks cannot turn a run red on findings. Only the policy can, and only once you enforce.
ctfkit itself opens no network connection and reads no cloud credentials. The stack's own init still needs your compliance.tf registry token to download the served modules that report reads, and its plan needs the stack's cloud credentials.
Prerequisites
- A Spacelift account where you can create stacks and plan policies, and set stack hooks and labels. Stack settings are a platform admin task on purpose: see Step 3.
- A container registry your Spacelift workers can pull from.
- ctfkit v0.2.1 or later. v0.2.0 added the per-stack label and the
after_planplan probe; v0.2.1 is the version this guide is written against. - Stacks that use compliance.tf served modules, with a registry token for your compliance.tf host available to
init, for example as the secret environment variableTF_TOKEN_bobthecorp_compliance_tffor the hostbobthecorp.compliance.tf. Terraform readsTF_TOKEN_<host>with the dots in the host name replaced by underscores, so set one variable per compliance.tf host in your module sources. CI/CD with other platforms shows how to set it on a stack or a context. - OpenTofu or Terraform on
PATHwhen theafter_planhook runs. The hook usestofuwhen it is there andterraformotherwise; see Step 1. - Optional: the
opaCLI, to check the policy offline before you attach it.
Step 1: build a runner image
compliance.tf publishes no Spacelift runner image. You build and host one yourself, which keeps the ctfkit version, the settings file and the exception registry under your control. Start from a Spacelift runner image: Spacelift hooks run through a shell.
The ctfkit releases are in a private repository, and an unauthenticated download returns 404. The Dockerfile below downloads the archive and checksums.txt through the GitHub API with a token passed as a BuildKit secret, so the token is never written to an image layer. Use a fine-grained personal access token from an account with access: resource owner compliancetf, repository ctfkit, permission Contents: read. The archive is checked against the release checksums before it is unpacked:
# syntax=docker/dockerfile:1
# Pin a tag or a digest you reviewed. Check the published tags in the public ECR gallery for
# spacelift/runner-terraform; do not build from :latest.
FROM public.ecr.aws/spacelift/runner-terraform:<pinned-tag>
ARG CTFKIT_VERSION=0.2.1
ARG TARGETARCH=amd64
USER root
# The base image is Alpine and has curl but not jq; the download below needs both.
RUN apk add --no-cache curl jq
# ctfkit: the Linux release archive, downloaded with the token from the gh_token build secret and
# checked against the release checksums before it is unpacked. Release archives are named
# ctfkit_<version>_Linux_x86_64.tar.gz and ctfkit_<version>_Linux_arm64.tar.gz.
RUN --mount=type=secret,id=gh_token set -eu; \
case "$TARGETARCH" in amd64) arch=x86_64 ;; arm64) arch=arm64 ;; *) exit 1 ;; esac; \
tarball="ctfkit_${CTFKIT_VERSION}_Linux_${arch}.tar.gz"; \
api="https://api.github.com/repos/compliancetf/ctfkit"; \
auth="Authorization: Bearer $(cat /run/secrets/gh_token)"; \
mkdir -p /tmp/ctfkit; cd /tmp/ctfkit; \
curl -fsSL -H "$auth" -H "Accept: application/vnd.github+json" \
"$api/releases/tags/v${CTFKIT_VERSION}" > release.json; \
for f in "$tarball" checksums.txt; do \
id=$(jq -r --arg n "$f" '.assets[] | select(.name == $n) | .id' release.json); \
[ -n "$id" ]; \
curl -fsSL -H "$auth" -H "Accept: application/octet-stream" \
-o "$f" "$api/releases/assets/$id"; \
done; \
awk -v a="$tarball" '$2==a' checksums.txt | sha256sum -c -; \
tar -xzf "$tarball" ctfkit; \
install -m 0755 ctfkit /usr/local/bin/ctfkit; \
cd /; rm -rf /tmp/ctfkit; \
ctfkit version
# Optional, for OpenTofu stacks where the image provides tofu instead of Spacelift: install a
# pinned OpenTofu release the same way, checked against its SHA256SUMS file.
# Settings and exceptions live in the image, never in the analyzed repository.
COPY ctfkit.yaml /opt/ctfkit/ctfkit.yaml
COPY exceptions.yaml /opt/ctfkit/exceptions.yaml
# Switch back to the base image's user: spacelift/runner-terraform runs as spacelift (uid 1983).
USER spaceliftNotes on the recipe:
- The two
/opt/ctfkit/paths are the ones every hook on this page passes with--configand--exceptions. Change them in both places or in neither. Anexceptions.yamlwith no entries suppresses nothing;ctfkit config initwrites a starterctfkit.yaml. Both formats are on Commands. - No catalog file is needed. The hooks pass
--catalog embedded, which pins the catalog compiled into the ctfkit version you installed. - OpenTofu: Spacelift can download and manage the OpenTofu or Terraform version for the stack, or you can bake a pinned binary into the image. Either works if
tofuorterraformis onPATHwhen theafter_planhook runs; the hook usestofuwhen both are there. If you rely on Spacelift's managed version, confirm that in the first run's log.
Build it with the token in GH_TOKEN, passed as the gh_token secret:
export GH_TOKEN="$CTFKIT_RELEASE_TOKEN" # from your secret store, never inline
docker build --secret id=gh_token,env=GH_TOKEN -t <your-registry>/ctfkit-runner:<tag> .Push it as <your-registry>/ctfkit-runner:<tag>, and use that name below.
Step 2: module repositories
A module repository published through the Spacelift module registry can run ctfkit report in every module test case. Copy config.yml below to .spacelift/config.yml at the root of the module repository and set:
runner_imageto your image,teststo the example roots your module ships.
These hooks live in the module repository because they only gate that repository's own releases. The plan policy in Step 4 does not apply to module test cases. The report hook starts at --fail-on never, so a test case never fails on findings while you read the first results. It renders receipt,spacelift, so its exit code decides the test case. To enforce, change it to --fail-on error: from then on an error-level finding fails the test case, and Spacelift publishes no module version.
# .spacelift/config.yml for a module repository served through the Spacelift module registry.
# Copy this to .spacelift/config.yml at the root of the module repository. These hooks gate that
# repository's own releases, so they belong beside its code. Consuming-stack hooks do not: see
# config.stack.yml and https://compliance.tf/docs/ctfkit/spacelift/.
#
# Every flag that decides an outcome is pinned here and reads from the runner image, never from the
# repository being analyzed: --fail-on, --format, --config, --exceptions, --catalog. A module
# repository therefore cannot lower its own threshold, widen its endpoint allowlist, grant itself an
# exception or swap in a different catalog by committing a file. `--catalog embedded` pins the
# release catalog compiled into the binary and ignores ~/.ctf/control-requirements.yaml, so
# control ids and remediation module sources always resolve and no catalog file has to be shipped
# or kept current.
#
# Shadow mode: --fail-on never means the hook always exits 0 on findings, so a test case can only
# fail on its own merits while the recipe is bedding in. To enforce, change it to --fail-on error.
# This hook renders more than one format, so its exit code decides the test case and the threshold
# matters here. The plan policy in policy.rego does not apply to module test cases.
version: "1"
test_defaults:
# An image that carries the pinned ctfkit binary on PATH, plus the settings and exception files
# the hooks read. The catalog needs no file: it is compiled into the binary. No runner image is
# published: build one on a Spacelift runner base with ctfkit v0.2.1 or later, following
# https://compliance.tf/docs/ctfkit/spacelift/, and push it to a registry your Spacelift workers
# can pull from. A distroless image has no shell and cannot run hooks, so it cannot be the runner
# image. A ctfkit container image is planned and not published.
runner_image: <your-registry>/ctfkit-runner:<tag>
after_init:
# A metadata file committed to the repository would otherwise be read as this run's result.
- rm -f ctfkit.custom.spacelift.json
- >-
ctfkit report
--fail-on never
--format receipt,spacelift --out-dir .ctfkit
--config /opt/ctfkit/ctfkit.yaml
--exceptions /opt/ctfkit/exceptions.yaml
--catalog embedded
# To import test results into the receipt, add --tests-json to the line above. ctfkit never
# runs tofu or terraform, so the file must already exist when this hook runs, written by the
# step that runs the tests, for example `tofu test -json > tests.json`:
# ctfkit report --fail-on never --format receipt,spacelift --out-dir .ctfkit --tests-json tests.json ...
tests:
- name: complete
project_root: examples/complete
- name: simple
project_root: examples/simpleStep 3: stack hooks
For a stack that consumes modules, set the hooks in the stack's settings, through the Spacelift UI, API or Terraform provider. Do not commit them as .spacelift/config.yml in the stack repository: anyone who can open a pull request could then remove the hook in the same change that adds the finding.
The diagram above shows what each hook runs and writes. The after_plan script:
- finds the saved plan: the first regular file among
spacelift.plan,*.planand*.tfplan, - turns it into
plan.jsonwithshow -json, usingtofuif it is onPATHandterraformotherwise, - runs
ctfkit scanon it as its last command.
With no plan file it logs one line and exits 0, and the policy reports the ctfkit-scan key as missing. Both hooks first remove the files an earlier run could have left, so the policy never reads a stale result.
The exact commands, as config.stack.yml ships them in v0.2.1. Set them in stack settings; do not commit this file:
# Hook configuration for a consuming stack.
#
# Configure this in the STACK'S SETTINGS, through the Spacelift UI, API or terraform provider, not
# as a .spacelift/config.yml in the stack repository. A file in the repository is editable by
# anyone who can open a pull request against it, so the hook that produces the evidence could be
# removed in the same change that introduces the finding. Stack settings are a platform admin
# concern and stay that way.
#
# The module-registry test hooks in config.yml are different: they live in the module repository's
# own .spacelift/config.yml, because they gate that repository's releases.
#
# Two hooks, two metadata keys, so neither overwrites the other. Each file is JSON with a verdict,
# a summary of counts and a findings list:
# after_init -> ctfkit report -> ctfkit.custom.spacelift.json
# after_plan -> ctfkit scan -> ctfkit-scan.custom.spacelift.json
#
# Every flag that decides an outcome is pinned here and reads from the runner image, never from the
# repository being analyzed: --fail-on, --format, --config, --exceptions, --catalog, so a stack
# cannot change any of them by committing a file. `--catalog embedded` pins the release catalog
# compiled into the binary and ignores ~/.ctf/control-requirements.yaml, so control ids and
# remediation module sources always resolve and no catalog file has to be shipped or kept current.
#
# Both hooks render a format list of exactly `spacelift`, which by contract exits 0 on findings, so
# neither can turn the run red however the threshold is set. Exit 2, a tool error, still fails the
# hook, and nothing here suppresses that. Whether a stack is enforced is decided in policy.rego, by
# enforce or by the ctfkit:enforce stack label.
version: "1"
# Your own image with ctfkit v0.2.1 or later on PATH; see https://compliance.tf/docs/ctfkit/spacelift/.
# A distroless image has no shell and cannot run hooks. A ctfkit container image is planned and not
# published.
runner_image: <your-registry>/ctfkit-runner:<tag>
after_init:
# A metadata file left in the working directory, by an earlier hook or committed to the
# repository, would otherwise be read by the policy as this run's result.
- rm -f ctfkit.custom.spacelift.json
- >-
ctfkit report
--fail-on error
--format spacelift
--config /opt/ctfkit/ctfkit.yaml
--exceptions /opt/ctfkit/exceptions.yaml
--catalog embedded
# A stack that runs tofu test can import the results: add --tests-json to the line above. ctfkit
# never runs tofu or terraform, so the file must already exist when this hook runs, written by
# the step that runs the tests, for example `tofu test -json > tests.json`:
# ctfkit report --fail-on error --format spacelift --tests-json tests.json ...
after_plan:
# `ctfkit scan` needs plan JSON, and ctfkit never runs tofu or terraform itself, so the hook turns
# the saved plan into JSON first, with tofu when it is on PATH and terraform otherwise. It takes
# the first regular file among spacelift.plan, *.plan and *.tfplan in the working directory. The
# plan file name is not yet verified on a live Spacelift run; the probe covers the
# likely names until it is. A stale ctfkit-scan file or plan.json is removed first, so the policy
# never reads a file this run did not write. With no plan file the hook logs one line to stderr
# and exits 0: the run carries no ctfkit-scan key, and the policy reports it missing, warned in
# shadow mode and denied once enforcing. A failing show exits 1, and ctfkit scan is the last
# command, so a ctfkit tool error (exit 2) fails the hook too.
- |
rm -f ctfkit-scan.custom.spacelift.json plan.json
p=
for f in spacelift.plan *.plan *.tfplan; do
if [ -f "$f" ]; then p=$f; break; fi
done
if [ -z "$p" ]; then
echo "ctfkit: no plan file (spacelift.plan, *.plan, *.tfplan) in $(pwd); scan skipped" >&2
else
if command -v tofu >/dev/null 2>&1; then tf=tofu; else tf=terraform; fi
"$tf" show -json "$p" > plan.json || exit 1
ctfkit scan --plan plan.json --stage post_plan \
--fail-on error \
--format spacelift \
--config /opt/ctfkit/ctfkit.yaml \
--exceptions /opt/ctfkit/exceptions.yaml \
--catalog embedded
fiCopy both hooks from this file when you set them.
Both hooks render only spacelift, and a run whose format list is only spacelift exits 0 on findings. A hook fails the run only on exit 2, a ctfkit tool or configuration error, or when show -json fails.
Step 4: plan policy, in shadow mode
Create a Spacelift Plan policy from policy.rego below and attach it to every stack that runs the hooks. The file is Rego v1 and carries import rego.v1, so it loads with either Spacelift engine_type, REGO_V0 or REGO_V1.
# Spacelift plan policy for ctfkit by compliance.tf.
#
# Attach this as a Plan policy to every stack that runs the ctfkit hooks. A consuming stack gets its
# hooks from its stack settings (see config.stack.yml), not from a committed file; a
# .spacelift/config.yml applies only to a module repository's test cases (see config.yml). The
# policy renders the verdict the hooks already computed and never re-derives one from counts.
package spacelift
# Rego v1 syntax. The import keeps the file loading on OPA 0.59 or later evaluating in v0 mode
# (0.59 added `import rego.v1`; an older OPA rejects it); on OPA 1.x it changes nothing.
import rego.v1
# Global switch. Set enforce := true to enforce every stack this policy is attached to. While it
# stays false, every rule below is a warning and nothing blocks, except on a stack carrying the
# ctfkit:enforce label (next rule). No hook can turn a run red on its own. Module repositories do
# not use this switch: their test cases switch by changing --fail-on in their .spacelift/config.yml.
#
# Enforce a stack once it has had several shadow runs whose warnings you have reviewed, every run
# carries both metadata keys, and no run reports an unexpected manifest_missing finding.
enforce := false
# Per-stack switch: a stack carrying the label ctfkit:enforce is enforced while enforce stays false,
# so a platform admin can move one stack at a time. Labels live in stack settings, so a stack cannot
# add or remove its own label from its own repository, unless its settings are managed from that
# same repository (an administrative stack or the Spacelift terraform provider). No label means
# shadow mode.
enforce_stack if "ctfkit:enforce" in input.spacelift.stack.labels
# Only the exact label enforces. Any other ctfkit: label is warned, so a typo such as
# ctfkit:enforced does not leave a stack in shadow mode unnoticed.
unrecognized_labels contains label if {
some label in input.spacelift.stack.labels
startswith(label, "ctfkit:")
label != "ctfkit:enforce"
}
warn contains msg if {
not enforcing
some label in unrecognized_labels
msg := sprintf("unrecognized ctfkit label %v; stack stays in shadow mode", [label])
}
warn contains msg if {
enforcing
some label in unrecognized_labels
msg := sprintf("unrecognized ctfkit label %v", [label])
}
enforcing if enforce
enforcing if enforce_stack
# Each hook writes its own metadata key, so after_init and after_plan never overwrite each other:
# `ctfkit report` writes ctfkit.custom.spacelift.json, `ctfkit scan` writes
# ctfkit-scan.custom.spacelift.json. A stack run is expected to carry both.
keys := {"ctfkit", "ctfkit-scan"}
schema := "ctfkit.spacelift/1"
report[key] := r if {
key := keys[_]
r := input.third_party_metadata.custom[key]
}
missing contains key if {
key := keys[_]
not input.third_party_metadata.custom[key]
}
# ctfkit computes the verdict when it writes the file: fail when any unsuppressed error-level
# finding exists. It does not depend on --fail-on, so a hook's threshold cannot make a failing run
# look clean. The policy reads that field rather than counting findings, which is also why a rule
# that runs at both stages cannot be double counted. The totals are quoted for the reader, never
# summed across the two files.
#
# Anything that is not exactly "pass" fails closed, including an absent verdict: a file this policy
# cannot read as a pass is not a pass.
verdict(r) := v if {
v := r.verdict
}
verdict(r) := "absent" if {
not r.verdict
}
complete_summary(r) if {
r.summary.error
r.summary.warning
}
counts(r) := c if {
complete_summary(r)
c := sprintf("%d error, %d warning", [r.summary.error, r.summary.warning])
}
counts(r) := "totals unavailable" if {
not complete_summary(r)
}
failed[key] := msg if {
r := report[key]
v := verdict(r)
v != "pass"
msg := sprintf("%v: verdict %v (%v)", [key, v, counts(r)])
}
# Shadow mode: surface everything, block nothing.
warn contains msg if {
missing[key]
msg := sprintf("ctfkit metadata is missing from this run under key %v", [key])
}
warn contains msg if {
msg := failed[_]
}
# Once enforcing, through enforce or the stack label, the policy fails closed. Metadata absent
# under either key, and metadata carrying a schema this policy does not know, both deny: a run
# where a hook silently did not execute cannot pass as a clean run.
deny contains msg if {
enforcing
missing[key]
msg := sprintf("ctfkit metadata is missing from this run under key %v", [key])
}
deny contains msg if {
enforcing
r := report[key]
r.schema != schema
msg := sprintf("ctfkit metadata under key %v is in unknown schema %v", [key, r.schema])
}
deny contains msg if {
enforcing
msg := failed[_]
}The policy ships with enforce := false. In that state it only warns:
- a
warnfor each file whose verdict is notpass, quoting that file's totals, - a
warnfor each missing key, so a hook that did not run is visible, - a
warn,unrecognized ctfkit label <label>; stack stays in shadow mode, for any stack label that starts withctfkit:and is not exactlyctfkit:enforce.
Leave it in shadow mode while you read the first runs. With repository access, you can check the policy offline first against the fixtures in adapters/spacelift/test/:
cd adapters/spacelift
opa eval --format pretty --data policy.rego --input test/scan-verdict-fail.json 'data.spacelift.warn'Step 5: enforce one stack at a time
Add the stack label ctfkit:enforce to a stack to enforce the policy on that stack only, while enforce stays false for every other stack. A stack without the label stays in shadow mode. The label must match exactly: any other label that starts with ctfkit:, such as a typo or ctfkit:enforce-later, does not enforce, and the policy warns unrecognized ctfkit label <label>; stack stays in shadow mode so the mistake is visible. On a stack that is already enforcing, the same warning drops the shadow-mode clause.
On an enforcing stack, warn keeps firing as before, and deny also fires when:
- either key is missing from the run,
- either file carries a schema the policy does not know,
- either file's verdict is anything other than
pass.
So a labeled stack with a failing verdict shows the warning and the denial together.
To roll a stack back to shadow mode, remove the label. Labels live in stack settings, so a stack cannot add or remove its own label by committing a file. That changes when the stack settings are managed from the same repository, through an administrative stack or the Spacelift Terraform provider: a change to that repository can then change the label. Manage stacks from a separate repository that only platform admins can change.
Before you enforce a stack
Enforce a stack when:
- it has had several shadow runs, and you have reviewed their warnings,
- every run carries both metadata keys,
ctfkitandctfkit-scan, - no run carries an unexpected error-level finding, that is, one you have not reviewed and either fixed or waived.
An enforced stack denies a run when either file's verdict is fail, which any unsuppressed error-level finding causes. The findings that can do that on a stack:
- from
after_plan:ctf.coverage.module_sourcefor a raw resource the catalog maps to a compliance.tf module, the error-levelctf.risk.sensitive_change_*rules, andctf.origin.provider_resource_type, - from
after_init:ctf.coverage.manifest_missingfor a module compliance.tf does not serve,ctf.coverage.posture_degraded, andctf.coverage.tests_failedwhen the hook imports test results.
A run is also denied when a key is missing. If the saved plan file name is not one the after_plan hook finds, ctfkit-scan is missing on every run.
An exception in the registry baked into the runner image can waive any of these findings. A finding with control ids needs a control entry for each of them, and a finding with none, such as manifest_missing, needs a rule entry. Set resource to the address the finding shows: a resource such as aws_s3_bucket.logs, or for a module instance its module address, such as module.vpc, or module.network.module.vpc when nested. Exceptions has the file format.
Once every stack you care about is enforcing, you can set enforce := true in the policy instead of labeling each stack. Enforcing is enforce OR the label, so a label does nothing extra once the flag is on, and removing a label no longer rolls that stack back.
Terraform example
The same setup with the spacelift-io/spacelift Terraform provider. It assumes three files sit next to it: policy.rego, and ctfkit-after-init.sh and ctfkit-after-plan.sh, which hold the after_init and after_plan hooks from config.stack.yml. Each file is one shell script, passed as a single list element. ctfkit-after-init.sh holds the rm -f line, then the folded ctfkit report command written on one line. ctfkit-after-plan.sh holds the after_plan block as it is. Keeping the hooks in files means you update them by copying the new release's text. The stack starts in shadow mode.
terraform {
required_providers {
spacelift = {
source = "spacelift-io/spacelift"
version = "~> 1.55"
}
}
}
resource "spacelift_policy" "ctfkit" {
name = "ctfkit"
description = "Renders the verdicts ctfkit by compliance.tf wrote in the after_init and after_plan hooks"
type = "PLAN"
engine_type = "REGO_V1"
body = file("${path.module}/policy.rego")
}
resource "spacelift_stack" "network" {
name = "network"
repository = "infrastructure"
branch = "main"
project_root = "network"
terraform_workflow_tool = "OPEN_TOFU"
runner_image = "<your-registry>/ctfkit-runner:<tag>"
# Shadow mode. To enforce this stack later, add the label:
# labels = ["ctfkit:enforce"]
labels = []
# The hooks of config.stack.yml, saved verbatim.
after_init = [
file("${path.module}/ctfkit-after-init.sh"),
]
after_plan = [
file("${path.module}/ctfkit-after-plan.sh"),
]
}
resource "spacelift_policy_attachment" "ctfkit_network" {
policy_id = spacelift_policy.ctfkit.id
stack_id = spacelift_stack.network.id
}Keep the policy, the stacks and their hooks in a stack that only platform admins can change. That is what stops a repository from removing its own hook or label.
Reading the results
Where to look on a run:
| Where | What you see |
|---|---|
The after_init and after_plan phases of the run log | ctfkit's stderr: a tool error, the no plan file line, the show -json error on failure |
| The plan policy evaluation on the run | the warn and deny messages below |
| The run's working directory | ctfkit.custom.spacelift.json and ctfkit-scan.custom.spacelift.json: JSON with a verdict, a summary of counts and a findings list |
Policy messages:
| Message | Means |
|---|---|
ctfkit: verdict fail (2 error, 1 warning) | ctfkit report found at least one unsuppressed error-level finding |
ctfkit-scan: verdict fail (...) | the same for ctfkit scan over the plan |
ctfkit-scan: verdict absent (...) | the file has no verdict field, which fails closed |
ctfkit metadata is missing from this run under key ctfkit-scan | the hook did not run or wrote nothing |
ctfkit metadata under key ctfkit is in unknown schema ... | the file comes from a ctfkit version this policy does not know (deny only) |
unrecognized ctfkit label ctfkit:enforce-later; stack stays in shadow mode | the stack carries a ctfkit: label other than exactly ctfkit:enforce, so it does not enforce; check the spelling |
Verdicts, per file:
| Verdict | When | Shadow mode | Enforcing |
|---|---|---|---|
pass | no unsuppressed error-level finding | nothing | nothing |
fail | at least one unsuppressed error-level finding | warn | warn and deny |
| absent or any other value | the file could not be read as a pass | warn | warn and deny |
| key missing | the hook did not write its file | warn | warn and deny |
unknown schema | the file is from a newer or older format | nothing | deny |
The verdict does not depend on --fail-on. Warnings and notes never make a verdict fail.
ctfkit exit codes, as the hooks see them:
| Code | Meaning | In a Spacelift hook |
|---|---|---|
0 | nothing at or above --fail-on | also what a --format spacelift run returns on findings |
1 | findings at or above --fail-on | only in the module test hook, which also renders receipt |
2 | tool or configuration error | fails the hook and the run |
The policy output is unsigned; signing is planned.
Troubleshooting
ctf.coverage.manifest_missing at error level. A module instance in .terraform/modules has no compliancetf-manifest.json. Either the module is not served by compliance.tf, or init fetched it from somewhere else. Check the module's source, and check that init used your compliance.tf registry token. A module instance with no compliancetf-manifest.json raises an error, unless its own directory declares no resource block (a helper of data sources, locals and outputs, such as hashicorp/dir/template). Such an instance creates nothing, so it is only listed. Child modules with resources are still reported.
ctfkit metadata is missing from this run under key ctfkit. The after_init hook did not run, or ran in a different directory. Check that the hook is set on the stack, that the runner image is yours, and that ctfkit is on PATH (ctfkit version in a before_init hook shows it).
... missing ... under key ctfkit-scan. Look for the hook's no plan file line in the after_plan log. If it is there, the saved plan has a name the hook does not look for. Run ls -la in an after_plan hook once to see the name, and report it to mail@compliance.tf: this is the open question that keeps the integration in preview. If the hook failed instead, the log shows why: usually neither tofu nor terraform is on PATH at that point (see Step 1), or show -json could not read the plan.
ctfkit metadata under key ... is in unknown schema .... The runner image's ctfkit writes a schema the attached policy.rego does not know. Update the policy from the same ctfkit release as the image. This only denies on enforcing stacks.
The hook fails with exit 2. ctfkit could not read its settings, exception registry or input. The first stderr line names the file. The usual cause is a missing /opt/ctfkit/ file in the image.
Limits and planned work
- Live Spacelift testing is pending. The hooks and the policy pass the offline tests, but the saved plan file name, and whether
tofuorterraformis onPATHatafter_plan, have not been confirmed on a live run. - No runner image is published. You build it from the recipe in Step 1.
- Signed output is planned. ctfkit does not sign its output today.
The same checks run in GitHub Actions; see Run checks in GitHub Actions.