Roll out and run in CI
This page takes the compliance.tf provider for AWS from warnings on every configuration to failed plans in CI, one configuration at a time. It is for platform leads who own the pipelines. Start after the Quickstart has switched at least one configuration.
Before you start
- The configurations you roll out already use
registry.compliance.tf/compliancetf/aws, and their state is rebound withterraform state replace-provider(Quickstart, step 3). - You can set environment variables in the plan and apply jobs of each pipeline.
How the modes decide
Every control has an action, deny or warn. The mode decides what a violated deny control does:
| Mode | Violated deny control | Violated warn control |
|---|---|---|
soft_fail (default) | Warning | Warning |
hard_fail | Error: the plan fails and the apply is not forwarded | Warning |
These rules qualify the table:
- The stricter setting wins. The mode comes from
enforcement_modein thecompliance {}block and from theCOMPLIANCETF_ENFORCEMENT_MODEvariable.soft_failin the configuration cannot lower ahard_failvariable. - An unchanged resource only warns. When an existing resource has no planned change (planned and prior state equal and fully known), a violated
denycontrol gives a warning at plan, even inhard_fail, with the note "No change planned: this violation warns until the resource changes; any change to it is blocked." Turning onhard_faildoes not fail the plan for resources that break a control and that you leave alone. The plan of any change to them fails. - Unknown values are checked twice. A value unknown until apply gives a
pendingwarning at plan, and the provider checks it again before apply.
After apply, a violation is a warning, except after an update, where it is an error; Terraform has already changed the resource at that point. On refresh or import, a violation is a warning.
Step 1: Start in soft_fail everywhere
Switch each configuration with the default mode and no compliance {} block. Findings are warnings, and no run fails on them. Plan every configuration and collect the warnings per configuration.
Step 2: Fix or record each finding
Each finding prints the control id, the resource and line, a Fix: line and a Docs: link:
Warning: [rds_db_instance_backup_retention_period_less_than_7] RDS backup retention period must be at least 7 days.
with aws_db_instance.orders,
on rds.tf line 12, in resource "aws_db_instance" "orders":
12: backup_retention_period = 1
Fix: Set backup_retention_period to 7 or more.
Docs: https://compliance.tf/docs/controls/aws/rds_db_instance_backup_retention_period_less_than_7/Set the value the Fix: line names. Some fixes, such as storage encryption on an existing RDS instance, make AWS replace the resource; read the plan before you apply it.
To print more or fewer lines after the snippet, such as the framework references, set COMPLIANCETF_DIAGNOSTICS. The values and fields are under Diagnostics.
Step 3: Turn on hard_fail in CI
When a configuration plans with no deny warnings, make violations fail its plan. Set the variable in the CI job:
# plan job
export COMPLIANCETF_ENFORCEMENT_MODE=hard_fail
terraform init -input=false
terraform plan -input=false -out=tfplan
# apply job, with the same COMPLIANCETF_* variables
terraform apply -input=false tfplan- Each Terraform run reads the variables itself, so set the same ones in the plan job and the apply job.
- With a saved plan, the provider checks each resource again before apply. A saved plan holds the configuration it was made from; after you change the configuration, plan again.
- The variable makes every configuration the job runs at least that strict, whatever its
compliance {}block says.
To pin the mode in the configuration itself, so it also applies on laptops:
provider "aws" {
region = "us-east-1"
compliance {
enforcement_mode = "hard_fail"
}
}An unencrypted RDS instance then stops the plan:
Error: [rds_db_instance_encryption_at_rest_enabled] RDS instances must have storage encryption enabled.
with aws_db_instance.non_compliant,
on main.tf line 25, in resource "aws_db_instance" "non_compliant":
25: storage_encrypted = false # deny
Fix: Set storage_encrypted = true.
Docs: https://compliance.tf/docs/controls/aws/rds_db_instance_encryption_at_rest_enabled/Step 4: Keep the evidence
Every check writes a record to .terraform/compliancetf/evidence, or to evidence_dir in the compliance {} block (COMPLIANCETF_EVIDENCE_DIR wins).
- A record holds the control, the resource type, the operation, the resource id and identity, the stage, the result and the mode.
- The resource address is not recorded, because a
for_eachkey can carry a value. A record alone does not name the module or thecountorfor_eachinstance. Other attribute values are not recorded. - Each line is hash-chained to the one before it. A line edited or removed in the middle of a file shows; a file cut short at the end does not. The record format is in the provider's architecture notes.
The files stay on the machine that ran Terraform, and a CI runner usually discards .terraform when the job ends. If you rely on them as evidence, copy them to write-once storage after each run.
Keep the caches
The provider caches the verified hashicorp/aws binary and its schema under compliancetf in the user cache directory: $XDG_CACHE_HOME or ~/.cache on Linux, ~/Library/Caches on macOS, %LocalAppData% on Windows. A hashicorp/aws of the pinned version in Terraform's plugin directories is used before any download.
In CI, persist this directory between jobs, or accept that each fresh runner downloads and verifies hashicorp/aws again. A restored directory must be owned by the job user and not writable by group or others, or the provider skips it with a warning. With the cache warm, a plan takes about as long as with hashicorp/aws directly; the performance notes have the measurements.
COMPLIANCETF_CACHE_DIR does not move this cache.
Check the result
- In a
hard_failjob, a pull request that adds a resource breaking adenycontrol fails atterraform plan, and the log names the control. - A pull request that leaves existing resources that break a control untouched still plans, with warnings.
- The evidence files of the run are in your write-once storage.
Findings you cannot fix yet
Exceptions are not available, and an exceptions_file is ignored with a warning; see Settings that are not available.
Keep the configuration that has the finding on soft_fail and make the others hard_fail. A CI job that sets COMPLIANCETF_ENFORCEMENT_MODE=hard_fail overrides soft_fail in the configuration, so leave the variable off for that configuration's job.
Upgrade the provider
Upgrading the provider upgrades hashicorp/aws to the same version (Versions). Keep the version constraint and lock file workflow you use today, and run terraform init -upgrade as before. A release can change the built-in controls, so read the CHANGELOG first.
Fixes to the compliance.tf layer alone ship as prereleases, which Terraform selects only when pinned exactly; see Versions.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
hard_fail in the block, but the plan only warns | The resource exists and has no planned change | Expected; any change to it fails the plan |
soft_fail in the block, but the plan fails | COMPLIANCETF_ENFORCEMENT_MODE=hard_fail is set in the job; the stricter setting wins | Leave the variable off for that job |
A pending warning at plan | A value is unknown until apply | The provider checks it again before apply |
| The cache is skipped with a warning | The restored directory is not owned by the job user, or group or others can write it | Restore it with the job user as owner and no group or other write bit |
enforcement_mode advisory is below the default soft_fail and has no effect | advisory is accepted and ignored | Use soft_fail or hard_fail |