Reference analyzers
ctfkit by compliance.tf ships a set of reference analyzers. Each one emits findings under stable rule IDs, at a fixed level, at one or more stages. This page lists every rule ID in the newest release, what makes it fire, and where it stops, so you can decide which findings to act on and which to leave to the tools you already run.
It is for platform engineers deciding which findings to enforce, and for anyone writing an OPA policy or an exception against a rule ID.
scanruns these families: module source coverage, disabled controls, provider and module origin, tagging, sensitive changes, and code that runs on the machine that plans or applies. Withwrapped_providersset, apre_planscan also runs the lock file rules.reportruns the coverage family over the module cache, and the code-execution rules over every moduleinitinstalled.verify-lockruns the lock file rules.- Levels:
error,warning,note.--fail-on(defaulterror) decides which levels fail ascanorreportrun. - A value that is unknown until apply never fires a rule; tagging reports it explicitly as
ctf.policy.tagging_unresolved. - Control ids and replacement module sources come from the control catalog: by default the one compiled into the binary, which holds the CIS AWS Benchmark v6.0 controls, or the full catalog file that
ctfkit catalog pulldownloads (Control catalog).ctfkit scanprints the catalog version on the summary line.
Rules at a glance
| Rule ID | Level | Stage | Command |
|---|---|---|---|
ctf.coverage.module_source | error or note | pre_plan, post_plan | scan |
ctf.coverage.module_source_unknown_registry | warning | pre_plan, post_plan | scan |
ctf.policy.disabled_control | error | pre_plan | scan |
ctf.policy.disabled_control_unknown | error | pre_plan | scan |
ctf.origin.provider | warning | pre_plan, post_plan | scan |
ctf.origin.provider_resource_type | error | pre_plan, post_plan | scan |
ctf.origin.provider_unpinned | warning | pre_plan | scan |
ctf.origin.provider_unverified | warning | pre_plan | scan |
ctf.policy.floating_ref | warning | pre_plan | scan |
ctf.policy.floating_ref_unbounded | warning | pre_plan | scan |
ctf.policy.tagging | warning | post_plan | scan |
ctf.policy.tagging_value | warning | post_plan | scan |
ctf.policy.tagging_unresolved | note | post_plan | scan |
ctf.risk.sensitive_change_public_exposure | error, some cases warning or note | post_plan | scan |
ctf.risk.sensitive_change_wildcard_iam | error or warning | post_plan | scan |
ctf.risk.sensitive_change_audit_log_removal | error, one case warning | post_plan | scan |
ctf.risk.sensitive_change_encryption_disablement | error | post_plan | scan |
ctf.risk.sensitive_change_destructive_stateful | error, some cases warning | post_plan | scan |
ctf.exec.provisioner | error | pre_plan, post_init | scan (pre_plan), report (post_init) |
ctf.exec.external_data | error | pre_plan, post_init | scan (pre_plan), report (post_init) |
ctf.exec.module_endpoint | error | post_init | report |
ctf.exec.module_uninspected | warning | pre_plan, post_init | scan (pre_plan), report (post_init) |
ctf.lock.missing | error | pre_plan | verify-lock, scan with wrapped_providers |
ctf.lock.upstream_present | error | pre_plan | verify-lock, scan with wrapped_providers |
ctf.lock.wrapper_missing | error | pre_plan | verify-lock, scan with wrapped_providers |
ctf.lock.wrapper_unhashed | error | pre_plan | verify-lock, scan with wrapped_providers |
ctf.lock.source_mismatch | error | pre_plan | verify-lock, scan with wrapped_providers |
ctf.lock.unwrapped | warning | pre_plan | verify-lock, scan with wrapped_providers |
ctf.coverage.manifest_missing | error | post_init | report |
ctf.coverage.posture_degraded | error | post_init | report |
ctf.coverage.tests_failed | error | post_init | report |
ctf.coverage.delegation_unresolved | warning | post_init | report |
ctf.coverage.obligation_unknown | note | post_init | report |
ctf.coverage.resource_unattested | note | post_init | report |
Module source coverage
Checks whether modules and resources come from compliance.tf, and names the compliance.tf module that replaces them. Runs at pre_plan and post_plan under the same rule ID.
| Rule ID | Level | Fires when |
|---|---|---|
ctf.coverage.module_source | error | a module source is an upstream terraform-aws-modules/* registry module. The fix is the same path on the framework host that --remediation-host selects, at the same version |
ctf.coverage.module_source | note | a module source is any other public registry module. Listed for inventory only |
ctf.coverage.module_source | error | post_plan only: a planned aws_* managed resource whose type a compliance.tf module covers in the catalog |
ctf.coverage.module_source_unknown_registry | warning | a registry address on a host that is not one of your compliance.tf endpoints |
Limits:
- Only module calls in the root module are checked. In
report,ctf.exec.module_endpointchecks the modules that other modules call. - Local paths,
git::ands3::sources, archive URLs and sources on a compliance.tf host stay silent. A compliance.tf source with?disable=belongs to the disabled-control family, so it is not reported twice. - The
allowed_endpointssetting decides which hosts count as compliance.tf. See Settings file. - The embedded catalog maps two resource types to a compliance.tf module:
aws_s3_buckettoterraform-aws-modules/s3-bucket/aws(~> 5.0) andaws_sqs_queuetoterraform-aws-modules/sqs/aws(~> 4.0). Other raw resources are not reported by this rule. Resources already inside a compliance.tf module are skipped. - On a raw resource, the finding lists only the controls the suggested module enforces, not every control the catalog places on that resource type.
- The suggested host comes from
--remediation-host(defaultsoc_2, givingsoc2.compliance.tf). It is always a framework host; see Hosts and registry tokens.
Disabled controls
Checks for controls switched off on a compliance.tf module source. Runs at pre_plan.
| Rule ID | Level | Fires when |
|---|---|---|
ctf.policy.disabled_control | error | a token in ?disable= on a compliance.tf module source is a control id in the catalog |
ctf.policy.disabled_control_unknown | error | a token in ?disable= is not a control id in the loaded catalog: a typo, or a control outside that catalog. With the embedded catalog, every control outside CIS v6.0 lands here; the finding carries no control id |
module "s3_bucket" {
source = "soc2.compliance.tf/terraform-aws-modules/s3-bucket/aws?disable=s3_bucket_enforces_ssl"
}Limits:
- Only sources whose host ends in
.compliance.tfare checked. ?disable=takes a comma-separated list and may repeat; each control is reported once per module call.- Every
.tf,.tofu,.tf.jsonand.tofu.jsonfile is read, root module or not.
Provider and module origin
Checks where providers and modules come from and how tightly they are pinned. These findings carry no control id; to waive one, write an exception with rule: instead of control: (see Exceptions).
| Rule ID | Stage | Level | Fires when |
|---|---|---|---|
ctf.origin.provider | pre_plan, post_plan | warning | a provider source is not in allowed_providers. An empty list allows every provider |
ctf.origin.provider_resource_type | pre_plan, post_plan | error | a resource type is not in allowed_resources. Active only when that list is set; data sources are not checked |
ctf.origin.provider_unpinned | pre_plan | warning | a required_providers entry has no version constraint, or none with an upper bound |
ctf.origin.provider_unverified | pre_plan | warning | a required_providers entry has no hashes in .terraform.lock.hcl. A configuration with no lock file at all is not reported |
ctf.policy.floating_ref | pre_plan | warning | a Git module source (git::, git@, github.com/, bitbucket.org/) has no ?ref=, or a ref that is not a full 40- or 64-character commit SHA |
ctf.policy.floating_ref_unbounded | pre_plan | warning | a registry module source has no version constraint with an upper bound |
A constraint is bounded when at least one of its comma-separated parts is an exact version or starts with =, ~>, < or <=. An empty constraint, or one made only of >, >= and != parts, is unbounded.
A reusable module declares the lowest provider version it supports, such as >= 6.0, and leaves the upper bound to the root that calls it. When scan reads a directory, ctf.origin.provider_unpinned therefore reports only the deployable roots in it, told apart from modules by their backend, cloud and provider blocks and by which directories call which. A file list, as the ctfkit pre-commit hook passes, is not classified: the rule reports every required_providers entry in the listed files that has no upper-bounded constraint. The ctfkit-scan-dir hook scans the repository as a directory. The full classification and its known misses are in Provider constraints in reusable modules in the ctfkit repository.
Tagging
Checks the effective tags on planned resources against the tags and required_tags settings. Runs at post_plan, and only when the settings file defines a tag policy.
| Rule ID | Level | Fires when |
|---|---|---|
ctf.policy.tagging | warning | a required tag key is missing |
ctf.policy.tagging_value | warning | a known tag value is not in allowed_values, or does not match pattern |
ctf.policy.tagging_unresolved | note | a constrained tag value, or the whole tags map, is unknown until apply |
Limits:
- Covers managed resources being created, updated or replaced. Resources with no
tagsortags_allattribute are out of scope. - Effective tags are the provider's constant
default_tags, thentags, thentags_all. - A rule with
resource_typesapplies only to those types. - At most one finding per resource per rule. Messages never contain a tag value.
- On
hashicorp/googleresources the rules judge labels:labels, the provider'sdefault_labelsandeffective_labelstake the place oftags. GCP label keys must be lower case, so a rule such asOwnercan never be met by a google resource; limit it to other types withresource_types, or add a lower-case rule for labels.
Sensitive changes
Checks the plan for changes that weaken security or destroy data. Runs at post_plan over managed resources being created, updated, deleted or replaced. An attribute that is unknown at plan time never fires. When the catalog maps the attribute to a compliance.tf control for that resource type, the finding carries the control and is marked as one a compliance.tf module prevents.
ctf.risk.sensitive_change_public_exposure
| Change | Level |
|---|---|
aws_s3_bucket_public_access_block deleted, or any of block_public_acls, block_public_policy, ignore_public_acls, restrict_public_buckets goes from true to false | error |
aws_s3_bucket_acl set to public-read or public-read-write | error |
a policy with principal * and no Condition in aws_s3_bucket_policy, aws_sns_topic_policy, aws_sqs_queue_policy, aws_ecr_repository_policy, aws_opensearch_domain_policy, or aws_iam_role assume_role_policy | error |
aws_lambda_permission with principal = "*" and no source_arn | error |
publicly_accessible turned on for aws_db_instance, aws_rds_cluster_instance, aws_redshift_cluster or aws_dms_replication_instance | error |
aws_instance associate_public_ip_address turned on | warning |
ingress from 0.0.0.0/0 or ::/0 on aws_security_group, aws_security_group_rule or aws_vpc_security_group_ingress_rule | error; note when the port range is exactly 80 or exactly 443 |
ctf.risk.sensitive_change_wildcard_iam
Checks the policy of aws_iam_policy, aws_iam_role_policy, aws_iam_user_policy and aws_iam_group_policy, and aws_iam_role inline_policy, for an Allow statement with no Condition on Resource: "*":
| Action | Level |
|---|---|
* | error |
a service-wide wildcard, such as s3:* | warning |
ctf.risk.sensitive_change_audit_log_removal
| Change | Level |
|---|---|
deleting aws_cloudtrail, aws_config_configuration_recorder, aws_config_delivery_channel, aws_flow_log or aws_s3_bucket_logging | error |
aws_cloudtrail enable_logging, include_global_service_events or is_multi_region_trail goes from true to false | error |
aws_guardduty_detector enable goes from true to false | error |
aws_eks_cluster enabled_cluster_log_types or aws_rds_cluster enabled_cloudwatch_logs_exports narrowed to a subset of its previous value | error |
aws_cloudwatch_log_group retention_in_days decreases, or changes from 0 (never expire) to a number | warning |
ctf.risk.sensitive_change_encryption_disablement
Every case is an error:
encryptedorstorage_encryptedturned off or removed onaws_ebs_volume,aws_db_instance,aws_rds_cluster,aws_efs_file_systemoraws_redshift_cluster.aws_dynamodb_tableserver_side_encryptionenabledremoved.aws_s3_bucket_server_side_encryption_configurationdeleted, or itsruleblock removed.aws_eks_clusterencryption_configremoved.kms_master_key_idemptied onaws_sqs_queueoraws_sns_topic.aws_kms_keyenable_key_rotationturned off.
ctf.risk.sensitive_change_destructive_stateful
Stateful types: aws_db_instance, aws_rds_cluster, aws_dynamodb_table, aws_s3_bucket, aws_efs_file_system, aws_elasticache_cluster, aws_elasticache_replication_group, aws_redshift_cluster, aws_docdb_cluster, aws_neptune_cluster, aws_opensearch_domain, aws_ebs_volume, aws_fsx_lustre_file_system, aws_fsx_windows_file_system, aws_msk_cluster, aws_kms_key, aws_secretsmanager_secret.
| Change | Level |
|---|---|
| delete or replace of a stateful resource; for a replacement, the message names the attributes that force it | error |
deletion_protection turned off on aws_db_instance, aws_rds_cluster, aws_docdb_cluster or aws_neptune_cluster, or deletion_protection_enabled turned off on aws_dynamodb_table | warning |
skip_final_snapshot turned on for aws_db_instance or aws_rds_cluster | warning |
The resource types and attributes these rules check, and the attributes that hold a resource's tags, come from versioned rule packs built into the binary, one per provider major version: hashicorp/aws v6 and a hashicorp/google v8 proof pack. The tables above are the AWS pack. When a plan changes resources of a provider no pack describes, scan names that provider in one stderr line and does not check those resources.
Code that runs on the machine that plans or applies
Before plan runs, the ctf.exec rules report code that would run on the machine that plans or applies the configuration: plan reads external data sources, and apply runs provisioners. Any configuration gets them, with or without a compliance.tf module. They carry no control id; to waive one, write an exception with rule:.
| Rule ID | Stage | Level | Fires when |
|---|---|---|---|
ctf.exec.provisioner | pre_plan, post_init | error | a resource or removed block declares a provisioner: local-exec, remote-exec, file or any other |
ctf.exec.external_data | pre_plan, post_init | error | a data "external" block, or another data source type the built-in rule pack lists as running a local program, at top level or inside a check block |
ctf.exec.module_endpoint | post_init | error | a module called from inside another module comes from a source allowed_endpoints does not admit. Only a ./ or ../ path, or a registry address on an allowed endpoint, is admitted |
ctf.exec.module_uninspected | pre_plan, post_init | warning | a module call whose files were not read: in scan, a ../ local module directory that does not exist yet; in report, a module block that .terraform/modules/modules.json has no record for |
Limits:
- A
pre_planscan of a directory reads the root, the local modules under it, and the./or../local modules it calls outside it, such as../shared. A file list is read as given. reportruns afterinitand also reads every moduleinitinstalled, at any depth, remote modules included.ctf.exec.module_endpointdoes not check a module the root calls directly:ctf.coverage.module_sourcechecks that source, so the two rules never report the same one.- A finding names the block's file, lines and address. It never quotes the command or the program, and every module source it quotes has its userinfo and query removed, since they can carry a token.
- A module directory, or a file in one, that resolves outside the configuration directory, other than through an explicit
../source, is exit2, not a finding. The rules for symlinks,modules.jsonand unreadable directories are in Code that runs before plan in the ctfkit repository.
Lock file rules
These rules check that the .terraform.lock.hcl of every deployable root records the compliance.tf provider wrapper address, registry.compliance.tf/compliancetf/<type>, and not the upstream hashicorp/<type> provider, for each wrapped type. ctfkit verify-lock [DIR] checks every deployable root under DIR (default: the working directory), for the types --provider names (repeatable: aws, azurerm, google), else those in wrapped_providers in the settings file, else aws. scan --stage pre_plan runs the same rules on the directory it scans, only when the settings file sets wrapped_providers, and for the types that key lists.
| Rule ID | Level | Fires when |
|---|---|---|
ctf.lock.missing | error | a root has no lock file |
ctf.lock.upstream_present | error | the lock has registry.terraform.io/hashicorp/<type> or registry.opentofu.org/hashicorp/<type> for a wrapped type |
ctf.lock.wrapper_missing | error | the lock has no registry.compliance.tf/compliancetf/<type> entry for a wrapped type the root uses |
ctf.lock.wrapper_unhashed | error | that entry records no zh: hash |
ctf.lock.source_mismatch | error | a required_providers entry, in a root or a module, names hashicorp/<type> for a wrapped type or names no source, or a resource or provider block of that type has no entry at all |
ctf.lock.unwrapped | warning | the lock has hashicorp/google-beta while google is wrapped |
Limits:
verify-lockhas no--fail-onand reads no exception registry: any finding, thectf.lock.unwrappedwarning included, exits1.- Serving the wrapper as
hashicorp/awsfrom a network mirror failsctf.lock.upstream_presentby design, because the lock keeps the upstream address. - A check that the wrapper entry matches the pinned wrapper release is planned.
Coverage (report)
ctfkit report runs these at post_init, over the compliancetf-manifest.json file that each compliance.tf served module carries in .terraform/modules. The receipt it writes is covered under report.
Terms used below:
- A served module is a module
initdownloaded from a compliance.tf host. - Its manifest,
compliancetf-manifest.json, records which controls the build enforces and what it does not prove. - A degraded build is one where the manifest records that part of your organization's posture could not be applied, with the reasons.
- A delegated control is one the module leaves to another scope, such as a separate module, to enforce.
- An obligation is something the manifest says the module does not prove, together with what you must do instead.
| Rule ID | Level | Fires when |
|---|---|---|
ctf.coverage.manifest_missing | error | a module instance has no compliancetf-manifest.json: it is not served by compliance.tf, or init fetched it from elsewhere |
ctf.coverage.posture_degraded | error | a served module's manifest records that the build was degraded; one finding per build |
ctf.coverage.tests_failed | error | the --tests-json input reports failed or errored tests |
ctf.coverage.delegation_unresolved | warning | a control is delegated to another scope, and no module instance in the configuration governs that scope. When the loaded catalog does not hold the control id, the message names it and the finding carries no control id; report still completes |
ctf.coverage.obligation_unknown | note | a manifest lists an obligation this ctfkit version does not know |
ctf.coverage.resource_unattested | note | the root module declares resources of its own, which no served module's manifest records |
Limits:
- A module instance whose own directory declares no
resourceblock, such as a helper of data sources, locals and outputs, is listed but not reported bymanifest_missing. - For
manifest_missing, the fix names a compliance.tf module when the instance's own resource types map to one in the catalog. A remote module that was never downloaded gets no fix. - A manifest in a schema major version this ctfkit does not support is an environment error, exit
2, not a finding. - No
--tests-jsoninput means notests_failedfinding.
What analyzers never do
- Evaluate HCL. At
pre_planonly literal values resolve. - Guess a value that is unknown until apply.
- Read live cloud state or call a provider.
post_applyis reserved and has no analyzers. - Run
terraformortofu, or open a network connection.scan,reportandverify-lockdo neither. The commandsgate planandruntask serverun thepost_plananalyzers on a plan they obtain themselves; Network and credentials lists what those commands run and call.
Related
- Commands: the flags, exit codes and exception file that act on these rule IDs.
- Quickstart: run a first scan and waive one finding.
- GitHub Actions and Spacelift: where these analyzers run in CI.