Sign in
Menu
Sign up
Docs
Check before applyctfkit

Reference analyzers

ctfkit by compliance.tf ships a set of reference analyzers. Each one emits findings under stable rule IDs, at a fixed level, at one or more stages. This page lists every rule ID in the newest release, what makes it fire, and where it stops, so you can decide which findings to act on and which to leave to the tools you already run.

It is for platform engineers deciding which findings to enforce, and for anyone writing an OPA policy or an exception against a rule ID.

  • scan runs these families: module source coverage, disabled controls, provider and module origin, tagging, sensitive changes, and code that runs on the machine that plans or applies. With wrapped_providers set, a pre_plan scan also runs the lock file rules.
  • report runs the coverage family over the module cache, and the code-execution rules over every module init installed.
  • verify-lock runs the lock file rules.
  • Levels: error, warning, note. --fail-on (default error) decides which levels fail a scan or report run.
  • A value that is unknown until apply never fires a rule; tagging reports it explicitly as ctf.policy.tagging_unresolved.
  • Control ids and replacement module sources come from the control catalog: by default the one compiled into the binary, which holds the CIS AWS Benchmark v6.0 controls, or the full catalog file that ctfkit catalog pull downloads (Control catalog). ctfkit scan prints the catalog version on the summary line.

Rules at a glance

Rule IDLevelStageCommand
ctf.coverage.module_sourceerror or notepre_plan, post_planscan
ctf.coverage.module_source_unknown_registrywarningpre_plan, post_planscan
ctf.policy.disabled_controlerrorpre_planscan
ctf.policy.disabled_control_unknownerrorpre_planscan
ctf.origin.providerwarningpre_plan, post_planscan
ctf.origin.provider_resource_typeerrorpre_plan, post_planscan
ctf.origin.provider_unpinnedwarningpre_planscan
ctf.origin.provider_unverifiedwarningpre_planscan
ctf.policy.floating_refwarningpre_planscan
ctf.policy.floating_ref_unboundedwarningpre_planscan
ctf.policy.taggingwarningpost_planscan
ctf.policy.tagging_valuewarningpost_planscan
ctf.policy.tagging_unresolvednotepost_planscan
ctf.risk.sensitive_change_public_exposureerror, some cases warning or notepost_planscan
ctf.risk.sensitive_change_wildcard_iamerror or warningpost_planscan
ctf.risk.sensitive_change_audit_log_removalerror, one case warningpost_planscan
ctf.risk.sensitive_change_encryption_disablementerrorpost_planscan
ctf.risk.sensitive_change_destructive_statefulerror, some cases warningpost_planscan
ctf.exec.provisionererrorpre_plan, post_initscan (pre_plan), report (post_init)
ctf.exec.external_dataerrorpre_plan, post_initscan (pre_plan), report (post_init)
ctf.exec.module_endpointerrorpost_initreport
ctf.exec.module_uninspectedwarningpre_plan, post_initscan (pre_plan), report (post_init)
ctf.lock.missingerrorpre_planverify-lock, scan with wrapped_providers
ctf.lock.upstream_presenterrorpre_planverify-lock, scan with wrapped_providers
ctf.lock.wrapper_missingerrorpre_planverify-lock, scan with wrapped_providers
ctf.lock.wrapper_unhashederrorpre_planverify-lock, scan with wrapped_providers
ctf.lock.source_mismatcherrorpre_planverify-lock, scan with wrapped_providers
ctf.lock.unwrappedwarningpre_planverify-lock, scan with wrapped_providers
ctf.coverage.manifest_missingerrorpost_initreport
ctf.coverage.posture_degradederrorpost_initreport
ctf.coverage.tests_failederrorpost_initreport
ctf.coverage.delegation_unresolvedwarningpost_initreport
ctf.coverage.obligation_unknownnotepost_initreport
ctf.coverage.resource_unattestednotepost_initreport

Module source coverage

Checks whether modules and resources come from compliance.tf, and names the compliance.tf module that replaces them. Runs at pre_plan and post_plan under the same rule ID.

Rule IDLevelFires when
ctf.coverage.module_sourceerrora module source is an upstream terraform-aws-modules/* registry module. The fix is the same path on the framework host that --remediation-host selects, at the same version
ctf.coverage.module_sourcenotea module source is any other public registry module. Listed for inventory only
ctf.coverage.module_sourceerrorpost_plan only: a planned aws_* managed resource whose type a compliance.tf module covers in the catalog
ctf.coverage.module_source_unknown_registrywarninga registry address on a host that is not one of your compliance.tf endpoints

Limits:

  • Only module calls in the root module are checked. In report, ctf.exec.module_endpoint checks the modules that other modules call.
  • Local paths, git:: and s3:: sources, archive URLs and sources on a compliance.tf host stay silent. A compliance.tf source with ?disable= belongs to the disabled-control family, so it is not reported twice.
  • The allowed_endpoints setting decides which hosts count as compliance.tf. See Settings file.
  • The embedded catalog maps two resource types to a compliance.tf module: aws_s3_bucket to terraform-aws-modules/s3-bucket/aws (~> 5.0) and aws_sqs_queue to terraform-aws-modules/sqs/aws (~> 4.0). Other raw resources are not reported by this rule. Resources already inside a compliance.tf module are skipped.
  • On a raw resource, the finding lists only the controls the suggested module enforces, not every control the catalog places on that resource type.
  • The suggested host comes from --remediation-host (default soc_2, giving soc2.compliance.tf). It is always a framework host; see Hosts and registry tokens.

Disabled controls

Checks for controls switched off on a compliance.tf module source. Runs at pre_plan.

Rule IDLevelFires when
ctf.policy.disabled_controlerrora token in ?disable= on a compliance.tf module source is a control id in the catalog
ctf.policy.disabled_control_unknownerrora token in ?disable= is not a control id in the loaded catalog: a typo, or a control outside that catalog. With the embedded catalog, every control outside CIS v6.0 lands here; the finding carries no control id
module "s3_bucket" {
  source = "soc2.compliance.tf/terraform-aws-modules/s3-bucket/aws?disable=s3_bucket_enforces_ssl"
}

Limits:

  • Only sources whose host ends in .compliance.tf are checked.
  • ?disable= takes a comma-separated list and may repeat; each control is reported once per module call.
  • Every .tf, .tofu, .tf.json and .tofu.json file is read, root module or not.

Provider and module origin

Checks where providers and modules come from and how tightly they are pinned. These findings carry no control id; to waive one, write an exception with rule: instead of control: (see Exceptions).

Rule IDStageLevelFires when
ctf.origin.providerpre_plan, post_planwarninga provider source is not in allowed_providers. An empty list allows every provider
ctf.origin.provider_resource_typepre_plan, post_planerrora resource type is not in allowed_resources. Active only when that list is set; data sources are not checked
ctf.origin.provider_unpinnedpre_planwarninga required_providers entry has no version constraint, or none with an upper bound
ctf.origin.provider_unverifiedpre_planwarninga required_providers entry has no hashes in .terraform.lock.hcl. A configuration with no lock file at all is not reported
ctf.policy.floating_refpre_planwarninga Git module source (git::, git@, github.com/, bitbucket.org/) has no ?ref=, or a ref that is not a full 40- or 64-character commit SHA
ctf.policy.floating_ref_unboundedpre_planwarninga registry module source has no version constraint with an upper bound

A constraint is bounded when at least one of its comma-separated parts is an exact version or starts with =, ~>, < or <=. An empty constraint, or one made only of >, >= and != parts, is unbounded.

A reusable module declares the lowest provider version it supports, such as >= 6.0, and leaves the upper bound to the root that calls it. When scan reads a directory, ctf.origin.provider_unpinned therefore reports only the deployable roots in it, told apart from modules by their backend, cloud and provider blocks and by which directories call which. A file list, as the ctfkit pre-commit hook passes, is not classified: the rule reports every required_providers entry in the listed files that has no upper-bounded constraint. The ctfkit-scan-dir hook scans the repository as a directory. The full classification and its known misses are in Provider constraints in reusable modules in the ctfkit repository.

Tagging

Checks the effective tags on planned resources against the tags and required_tags settings. Runs at post_plan, and only when the settings file defines a tag policy.

Rule IDLevelFires when
ctf.policy.taggingwarninga required tag key is missing
ctf.policy.tagging_valuewarninga known tag value is not in allowed_values, or does not match pattern
ctf.policy.tagging_unresolvednotea constrained tag value, or the whole tags map, is unknown until apply

Limits:

  • Covers managed resources being created, updated or replaced. Resources with no tags or tags_all attribute are out of scope.
  • Effective tags are the provider's constant default_tags, then tags, then tags_all.
  • A rule with resource_types applies only to those types.
  • At most one finding per resource per rule. Messages never contain a tag value.
  • On hashicorp/google resources the rules judge labels: labels, the provider's default_labels and effective_labels take the place of tags. GCP label keys must be lower case, so a rule such as Owner can never be met by a google resource; limit it to other types with resource_types, or add a lower-case rule for labels.

Sensitive changes

Checks the plan for changes that weaken security or destroy data. Runs at post_plan over managed resources being created, updated, deleted or replaced. An attribute that is unknown at plan time never fires. When the catalog maps the attribute to a compliance.tf control for that resource type, the finding carries the control and is marked as one a compliance.tf module prevents.

ctf.risk.sensitive_change_public_exposure

ChangeLevel
aws_s3_bucket_public_access_block deleted, or any of block_public_acls, block_public_policy, ignore_public_acls, restrict_public_buckets goes from true to falseerror
aws_s3_bucket_acl set to public-read or public-read-writeerror
a policy with principal * and no Condition in aws_s3_bucket_policy, aws_sns_topic_policy, aws_sqs_queue_policy, aws_ecr_repository_policy, aws_opensearch_domain_policy, or aws_iam_role assume_role_policyerror
aws_lambda_permission with principal = "*" and no source_arnerror
publicly_accessible turned on for aws_db_instance, aws_rds_cluster_instance, aws_redshift_cluster or aws_dms_replication_instanceerror
aws_instance associate_public_ip_address turned onwarning
ingress from 0.0.0.0/0 or ::/0 on aws_security_group, aws_security_group_rule or aws_vpc_security_group_ingress_ruleerror; note when the port range is exactly 80 or exactly 443

ctf.risk.sensitive_change_wildcard_iam

Checks the policy of aws_iam_policy, aws_iam_role_policy, aws_iam_user_policy and aws_iam_group_policy, and aws_iam_role inline_policy, for an Allow statement with no Condition on Resource: "*":

ActionLevel
*error
a service-wide wildcard, such as s3:*warning

ctf.risk.sensitive_change_audit_log_removal

ChangeLevel
deleting aws_cloudtrail, aws_config_configuration_recorder, aws_config_delivery_channel, aws_flow_log or aws_s3_bucket_loggingerror
aws_cloudtrail enable_logging, include_global_service_events or is_multi_region_trail goes from true to falseerror
aws_guardduty_detector enable goes from true to falseerror
aws_eks_cluster enabled_cluster_log_types or aws_rds_cluster enabled_cloudwatch_logs_exports narrowed to a subset of its previous valueerror
aws_cloudwatch_log_group retention_in_days decreases, or changes from 0 (never expire) to a numberwarning

ctf.risk.sensitive_change_encryption_disablement

Every case is an error:

  • encrypted or storage_encrypted turned off or removed on aws_ebs_volume, aws_db_instance, aws_rds_cluster, aws_efs_file_system or aws_redshift_cluster.
  • aws_dynamodb_table server_side_encryption enabled removed.
  • aws_s3_bucket_server_side_encryption_configuration deleted, or its rule block removed.
  • aws_eks_cluster encryption_config removed.
  • kms_master_key_id emptied on aws_sqs_queue or aws_sns_topic.
  • aws_kms_key enable_key_rotation turned off.

ctf.risk.sensitive_change_destructive_stateful

Stateful types: aws_db_instance, aws_rds_cluster, aws_dynamodb_table, aws_s3_bucket, aws_efs_file_system, aws_elasticache_cluster, aws_elasticache_replication_group, aws_redshift_cluster, aws_docdb_cluster, aws_neptune_cluster, aws_opensearch_domain, aws_ebs_volume, aws_fsx_lustre_file_system, aws_fsx_windows_file_system, aws_msk_cluster, aws_kms_key, aws_secretsmanager_secret.

ChangeLevel
delete or replace of a stateful resource; for a replacement, the message names the attributes that force iterror
deletion_protection turned off on aws_db_instance, aws_rds_cluster, aws_docdb_cluster or aws_neptune_cluster, or deletion_protection_enabled turned off on aws_dynamodb_tablewarning
skip_final_snapshot turned on for aws_db_instance or aws_rds_clusterwarning

The resource types and attributes these rules check, and the attributes that hold a resource's tags, come from versioned rule packs built into the binary, one per provider major version: hashicorp/aws v6 and a hashicorp/google v8 proof pack. The tables above are the AWS pack. When a plan changes resources of a provider no pack describes, scan names that provider in one stderr line and does not check those resources.

Code that runs on the machine that plans or applies

Before plan runs, the ctf.exec rules report code that would run on the machine that plans or applies the configuration: plan reads external data sources, and apply runs provisioners. Any configuration gets them, with or without a compliance.tf module. They carry no control id; to waive one, write an exception with rule:.

Rule IDStageLevelFires when
ctf.exec.provisionerpre_plan, post_initerrora resource or removed block declares a provisioner: local-exec, remote-exec, file or any other
ctf.exec.external_datapre_plan, post_initerrora data "external" block, or another data source type the built-in rule pack lists as running a local program, at top level or inside a check block
ctf.exec.module_endpointpost_initerrora module called from inside another module comes from a source allowed_endpoints does not admit. Only a ./ or ../ path, or a registry address on an allowed endpoint, is admitted
ctf.exec.module_uninspectedpre_plan, post_initwarninga module call whose files were not read: in scan, a ../ local module directory that does not exist yet; in report, a module block that .terraform/modules/modules.json has no record for

Limits:

  • A pre_plan scan of a directory reads the root, the local modules under it, and the ./ or ../ local modules it calls outside it, such as ../shared. A file list is read as given.
  • report runs after init and also reads every module init installed, at any depth, remote modules included.
  • ctf.exec.module_endpoint does not check a module the root calls directly: ctf.coverage.module_source checks that source, so the two rules never report the same one.
  • A finding names the block's file, lines and address. It never quotes the command or the program, and every module source it quotes has its userinfo and query removed, since they can carry a token.
  • A module directory, or a file in one, that resolves outside the configuration directory, other than through an explicit ../ source, is exit 2, not a finding. The rules for symlinks, modules.json and unreadable directories are in Code that runs before plan in the ctfkit repository.

Lock file rules

These rules check that the .terraform.lock.hcl of every deployable root records the compliance.tf provider wrapper address, registry.compliance.tf/compliancetf/<type>, and not the upstream hashicorp/<type> provider, for each wrapped type. ctfkit verify-lock [DIR] checks every deployable root under DIR (default: the working directory), for the types --provider names (repeatable: aws, azurerm, google), else those in wrapped_providers in the settings file, else aws. scan --stage pre_plan runs the same rules on the directory it scans, only when the settings file sets wrapped_providers, and for the types that key lists.

Rule IDLevelFires when
ctf.lock.missingerrora root has no lock file
ctf.lock.upstream_presenterrorthe lock has registry.terraform.io/hashicorp/<type> or registry.opentofu.org/hashicorp/<type> for a wrapped type
ctf.lock.wrapper_missingerrorthe lock has no registry.compliance.tf/compliancetf/<type> entry for a wrapped type the root uses
ctf.lock.wrapper_unhashederrorthat entry records no zh: hash
ctf.lock.source_mismatcherrora required_providers entry, in a root or a module, names hashicorp/<type> for a wrapped type or names no source, or a resource or provider block of that type has no entry at all
ctf.lock.unwrappedwarningthe lock has hashicorp/google-beta while google is wrapped

Limits:

  • verify-lock has no --fail-on and reads no exception registry: any finding, the ctf.lock.unwrapped warning included, exits 1.
  • Serving the wrapper as hashicorp/aws from a network mirror fails ctf.lock.upstream_present by design, because the lock keeps the upstream address.
  • A check that the wrapper entry matches the pinned wrapper release is planned.

Coverage (report)

ctfkit report runs these at post_init, over the compliancetf-manifest.json file that each compliance.tf served module carries in .terraform/modules. The receipt it writes is covered under report.

Terms used below:

  • A served module is a module init downloaded from a compliance.tf host.
  • Its manifest, compliancetf-manifest.json, records which controls the build enforces and what it does not prove.
  • A degraded build is one where the manifest records that part of your organization's posture could not be applied, with the reasons.
  • A delegated control is one the module leaves to another scope, such as a separate module, to enforce.
  • An obligation is something the manifest says the module does not prove, together with what you must do instead.
Rule IDLevelFires when
ctf.coverage.manifest_missingerrora module instance has no compliancetf-manifest.json: it is not served by compliance.tf, or init fetched it from elsewhere
ctf.coverage.posture_degradederrora served module's manifest records that the build was degraded; one finding per build
ctf.coverage.tests_failederrorthe --tests-json input reports failed or errored tests
ctf.coverage.delegation_unresolvedwarninga control is delegated to another scope, and no module instance in the configuration governs that scope. When the loaded catalog does not hold the control id, the message names it and the finding carries no control id; report still completes
ctf.coverage.obligation_unknownnotea manifest lists an obligation this ctfkit version does not know
ctf.coverage.resource_unattestednotethe root module declares resources of its own, which no served module's manifest records

Limits:

  • A module instance whose own directory declares no resource block, such as a helper of data sources, locals and outputs, is listed but not reported by manifest_missing.
  • For manifest_missing, the fix names a compliance.tf module when the instance's own resource types map to one in the catalog. A remote module that was never downloaded gets no fix.
  • A manifest in a schema major version this ctfkit does not support is an environment error, exit 2, not a finding.
  • No --tests-json input means no tests_failed finding.

What analyzers never do

  • Evaluate HCL. At pre_plan only literal values resolve.
  • Guess a value that is unknown until apply.
  • Read live cloud state or call a provider. post_apply is reserved and has no analyzers.
  • Run terraform or tofu, or open a network connection. scan, report and verify-lock do neither. The commands gate plan and runtask serve run the post_plan analyzers on a plan they obtain themselves; Network and credentials lists what those commands run and call.

On this page

Ask AI about this

Help improve this page